Papers with fine tuning
Deep Bidirectional Transformers for Relation Extraction without Supervision (D19-61)
Copied to clipboard
| Challenge: | Existing frameworks for relation extraction use distant supervision instead of annotated data. |
| Approach: | They propose a framework to deal with relation extraction tasks without supervision . they use syntactic parsing and pre-trained word embeddings to extract relations . |
| Outcome: | The proposed framework outperforms baselines on four biomedical datasets and achieves slightly worse results than the state-of-the-art in three out of four data sets. |
Meta-learning Pathologies from Radiology Reports using Variance Aware Prototypical Networks (2022.emnlp-industry)
Copied to clipboard
| Challenge: | Existing methods for fewshot learning require a large number of in-domain labeled examples for fine tuning. |
| Approach: | They propose to extend the Prototypical Networks for few-shot text classification by replacing Gaussian class prototypes with a regularization term that encourages the examples to be clustered near the appropriate class centroids. |
| Outcome: | The proposed method outperforms baselines on 13 public and 4 internal datasets and detects potential out-of-distribution (OOD) data points during deployment. |
Exploring Mathematical Extrapolation of Large Language Models with Synthetic Data (2024.findings-acl)
Copied to clipboard
| Challenge: | Large language models (LLMs) have shown excellent capabilities in language understanding, text generation and many other tasks, but struggle in complex multi-step reasoning problems such as mathematical reasoning. |
| Approach: | They propose to fine tune an open-llama-3B model to perform well on multi-step reasoning tasks via synthetic data. |
| Outcome: | The proposed model can reach a zero-shot pass@1 at 0.44 on the in-domain dataset and demonstrates certain generalization capabilities on the out-of-domain data. |
NLP Service APIs and Models for Efficient Registration of New Clients (2020.findings-emnlp)
Copied to clipboard
| Challenge: | State-of-the-art NLP inference uses enormous neural architectures and models trained for GPU-months, well beyond the reach of most consumers of NLP. |
| Approach: | They propose a centralized NLP service that can be customized to suit clients . they propose NER, sentiment labeling, and predictive language modeling to improve client experience. |
| Outcome: | The proposed model can be used to improve word usage and salience across clients without re-training or fine-tuning. |
Pyramid-BERT: Reducing Complexity via Successive Core-set based Token Selection (2022.acl-long)
Copied to clipboard
| Challenge: | Existing models that use heuristics to shorten sequence lengths are computationally prohibitive. |
| Approach: | They propose a new method to shorten sequence lengths by transforming tokens through encoders and a core-set based token selection method that avoids expensive pre-training and fine tuning. |
| Outcome: | The proposed model outperforms existing models on GLUE benchmarks and Long Range Arena datasets and demonstrates that it is cost-effective and space-efficient. |